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ABSTRACT 



This report quantitatively shows the progress of students in 
the area of general education development at College of DuPage (Illinois) . 

The aims of general education are to promote students ' understanding and 
appreciation of their culture and environment; to develop a system of 
personal values based on accepted ethics that lead to civic and social 
responsibility; and to attain skills in analysis, communication, and 
quantification, leading to development of lifelong learning. In response to 
assessment challenges, and to provide a means of comparison, the college 
randomly selected students to take one of the six College Assessment of 
Academic Proficiency (CAAP) area tests. A total of 1154 tests were 
administered to students at the beginning and towards the end of their 
studies. Slightly more than half of the examinees identified themselves as 
female, 76 percent were enrolled full-time, and roughly half of the students 
indicated that their primary goal was transferring to a four year university. 
Compared with national community college norms, the freshman appeared to be 
significantly higher in general education skills of essay writing and 
mathematics, but no different in writing, reading, critical thinking and 
science reasoning skills. DuPage sophomore students score significantly 
higher in mathematics and science reasoning but lower in reading than 
community college students nationally. Statistical tables and test models are 
included. (AF) 
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/assessment of general education at an institutional level at College of DuPage, or any community 
college, is a challenging task. Consideration of the challenges can be divided into at (east two distinct 
sets of issues. One set of issues focuses on defining general education and then operationalizing that 
assessment. A second set of issues focuses on designing and then operationalizing the measures 
selected in the first focus. Responding to these challenges, the General Education sub-committee of the 
Student Outcome Assessment Committee and the General Education Enduring Purposes Committee 
memberships have cooperated during the past two years to complete a first round of representative 
testing of general education and to provide feedback to all constituencies concerning their findings. 

Challenges to assessment at College of DuPage 

As with any inquiry, the first steps involve identifying what one intends to study. Within the general 
assessment framework, student learning needs to be examined through multiple measures at multiple 
levels from each classroom up through institutional levels. Activities at classroom and disciplinary levels 
have been initiated and are described in other documents of the Student Outcome Assessment 
Committee. When considering general education assessment, two limits need to be addressed: what are 
the skills, attitudes, and characteristics being assessed and who is eligible within the population as 
representative of the cohort being assessed? 

Answering the first question, “What is general education?" is a challenge faced at any institution 
attempting general education assessment. Discussion and debate as to a meaning for “general 
education” can occupy years and careers. A choice made by members of the Student Outcome 
Assessment Committee was to operationalize some inquiry, accepting that future discussions and 
alternative measures might result in other assessment formats. Thus, this round of assessment of 
general education at College of DuPage utilized the following premises. 

► General Education is learning which occurs as a result of multiple experiences and courses 
throughout a student's educational activities, not a specific learning resulting from a sequence of 
identified courses. 

► General Education is defined as those ideals expressed in the college catalog. 

The aims of general education are to enable students to 
understand and appreciate their culture and environment; to 
develop a system of personal values based on accepted ethics that 
lead to civic and social responsibility; and to attain the skills in 
analysis, communication, quantification, and synthesis necessary 
for further growth as a lifespan-learner and productive member of 
society. 

Consideration of this published statement results in identification of seven competencies that may be 
taken as an operational definition of general education at College of DuPage. 

1 . An aim of general education is to enable students to understand and appreciate their 

culture 

2. An aim of general education is to enable students to understand and appreciate their 

environment 

3. An aim of general education is to develop a system of personal values based on 

accepted ethics that lead to civic and social responsibility 

4. * An aim of general education is to attain the skills in analysis (critical thinking, scientific 

reasoning) necessary for further growth as a lifespan-learner and productive member of 
society. 

5. * An aim of general education is to attain the skills in communication necessary for further 

growth as a lifespan-learner and productive member of society. 

6* An aim of general education is to attain the skills in quantification necessary for further 
growth as a lifespan-learner and productive member of society. 

7* An aim of general education is to attain the skills in synthesis necessary for further growth 
as a lifespan-learner and productive member of society. 
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It should be evident that there is no single procedure adequate to assess all of these outcomes. 

However, there is a core of general education competencies which can be identified as academic skills 
and which can be assessed using nationally standardized tests. These skills are analysis, 
communication, and quantification, and they are marked with *'s above. We will return to 
operationalizing this definition of general education later. Continuing with defining the project, one 
moves to consider identification of the population. 

At many colleges and universities one can clearly identify students as admitted to the college or 
program, graduating or finishing a program, etc. Some of these same criteria are used at College of 
DuPage when students in programs are clearly admitted and completed. Such is not, however, the 
situation with general education. Students come for a few courses, come to complete degrees, come to 
earn enough credit to transfer, etc. 

Along with this lack of clearly identified qualification, past efforts at the college to recruit students’ 
participation in testing have failed despite rewards including free course work and bookstore coupons. (It 
may be that the idea of testing is the issue, given students’ willingness to participate in focus groups 
when offered pizza and pop.) 

Responses to the assessment challenges: 

Designing a response to these challenges resulted in applying a process first developed by the Value 
Added Committee in 1996. It was that committee which first designed a random section based sampling 
used to field test use of the six CAAP area-tests. 

Why the CAAP tests? 

Since one goal of general education assessment was to provide a comparison to other institutions, 
selection from among three national standardized tests of general skills was undertaken rather than the 
development of a local college specific test of minimum competence. Members of the Student Outcome 
Assessment Committee chose the College Assessment of Academic Proficiency (CAAP) based on four 
observations. 

► CAAP provided national norms for both 2-year and 4-year colleges of different categories. 

► CAAP provided unique area tests which could be aggregated as valid and reliable institution 
bases. This feature contrasted with more global single tests from which subject area scores were 
derived. 

► Each of these area tests could be administered in 50 minutes (according to the test protocol 
documents). 

► CAAP was judged least dependent on specific content ideas and, therefore, was judged most 
likely to evaluate general learning when students were not mandated into specific core courses. 

The six area-tests lined up well with a general education focus. Analysis was assessed with the area- 
tests of Critical Thinking, and Science Reasoning. Communication was assessed with Writing Skills (a 
multiple choice test), Essay Writing ( a demonstrated essay), and college level Reading in both the arts 
and social sciences area-tests. Quantification was assessed using a Math test which covered material 
from algebra through calculus. With the choice of evaluation tool in place, the next issue was student- 
subject selection. Assessment of synthesis was integral to each of the six area-tests. 

Sampling model: 

Because mandatory college wide testing was judged as not feasible and voluntary testing efforts of the 
past had resulted in high recruiting costs for low participation, a sampling model utilizing a stratified, 
random sampling of class-sections was developed. The random selection of class-sections assured 
testing a broad range of students with relatively low non-participation. But, as may be evident and 
perhaps already distracting to the reader, such testing will result in the assessment of a range of 
students, only a few of whom may match the ideal characteristics of entering freshmen or completing, 
graduating or transferring sophomores. Utilizing this full range of students’ data is an issue we 
considered in designing the analysis of these data. The design of the analysis will be discussed later 
(see page 3). For now let’s finish the explanation of the sampling mode. 

Assessment is conducted twice a year - fall and spring. In each round of assessment we attempted to 
gather at least 100 tests in each area. Thus, approximately 700 tests are administered in each testing 
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cycle. The model called for random sampling from all possible sections of introductory classes (100 
level single classes and sequences classes ending in 1 during the fall. During the spring the sampling 
was drawn from 200 level courses and end of sequences courses. In all selected sections, each of the 
six area-tests was rotationally assigned to some students. Thus, in any one section students would be 
completing each of the area-tests. 

Analysis model - Questions being addressed 

Given the challenges of defining and administering a simple pre-post test model at College of DuPage, 
the model for analysis must also be designed to accommodate the constraints. If one accepts that a 
clear pre-post test model cannot be implemented, but that a representative sampling of all students can 
be drawn using the data collection method described, then several alternative analysis models can be 
designed. 

Coming closest to traditional pre-post test modeling is comparing a cross-section of students beginning 
their studies with another cohort near completion of their studies. Such a comparison can focus on two 
questions. First, are the entering and leaving students similar to national standards for similar students 
at other colleges and universities? Second, is there an indication that students have changed during 
their studies at an institution? 

In the report section these types of models are reported (see page 13). However, it needs to be noted 
that such models are limited in the number of cases which can be classified as appropriate to the two 
points used in the model. 

An alternative analysis lies in constructing a statistical model which attempts to predict general education 
skills development based on a continuous development over the number and type of courses taken. 

Such a model can use most of the data collected in that it focuses on more than the initial and final 
points on the conidium of learning. Such models, although more complex than the two point analysis, 
can be very interesting. These models are reported starting on page 13. 

The characteristics of students taking one of the CAAP area-tests during the 1 999 academic year are 
summarized below, followed by comparisons of College of DuPage students with national norms starting 
on page 6. 



Report findings: 

General student characteristics: 

A total of 1154 tests were completed during the fall and spring testing cycle. During the fall of 1998 
twenty-seven course sections were sampled, and during the spring of 1999 thirty-two course sections 
were sampled. The number of tests in each of the six area-tests are roughly equivalent. 



Frequencies of subject-area tests by 
quarter 


Fall, 1998 
(1999.1) 


Spring, 1999 
(1999.3) 


Total 


Writing Skills 


111 


84 


195 


Math 


96 


92 


188 


Reading 


100 


95 


195 


Critical Thinking 


98 


91 


189 


Science Reasoning 


95 


94 


189 


Essav 


109 


89 


198 


Total Tests Completed 


609 


545 


1154 


Number of Course Sections Sampled 


27 


32 
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Fifty-five percent of these test takers identified themselves as female. 



Sex 


Frequency 


Percent 


Female 


628 


54.8 


Male 


518 


45.2 


Total 


1146 


100.0 



Seventy-six percent reported being enrolled full-time for the quarter during which they completed the 
CAAP assessment. 



Enrollment status 


Frequency 


Percent 


Full-Time 


865 


76.3 


Part-Time 


269 


23.7 


Total 


1134 


100.0 



The goal most frequently selected as most important was transferring to a college or university with over 
one-half of the students selecting this option. At a substantially lower level was receiving a degree or 
certificate. Given this focus on transfer, it is not surprising that we have a difficult time finding students 
who can be traditionally defined as entering as freshmen, completing two years of studies, then leaving 
as sophomores having completed some degree. 



Most important goal 


Frequency 


Percent 


Associates Degree 


390 


37.6 


Certificate in specific field 


36 


3.5 


Transfer to college or university 


536 


51.7 


Transfer to technical school 


9 


.9 


Upgrade current job skills 


10 


1.0 


Learn a specific skill 


13 


1.3 


Personal Interest 


16 


1.5 


Other 


26 


2.5 


Total ^ 


1036 


100.0 



Issues of time availability are a reoccurring theme among community college students. When asked 
how many hours per week they spent studying for their average class, the most frequent response was 0. 
Seventeen percent of the students reported they spent no time on their average course outside of class 
attendance. The average study time reported was 3.8 hours with 4 hours being the median reported time 
(s.d. = 2.85). The issue of study time will be examined as it relates to development of general education 
skills starting on page 22. 



Average hours per week study time per class 




Frequency 


Percent 




Frequency 


Percent 


0 


177 


17.2 


5 


121 


11.7 


1 


90 


8.7 


6 


74 


7.2 


2 


111 


10.8 


7 


47 


4.6 


3 


131 


12.7 


8 


57 


5.5 


4 


126 


12.2 


9 


98 


9.5 




Total 


1032 


100.0 



Two time demands which often competed with studying were employment and household commitments. 
The question about employment was expanded to include commitments to volunteer work hours. The 
median reported employment/volunteer commitments was between 16 and 30 hours. The median 
reported household work was between 1 and 15 hours. 
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Time commitments 


Employment (volunteer) 


Household work 


Frequency 


Percent 


Frequency 


Percent 


None 


242 


23.4 


210 


20.3 


1-15 hours per week 


154 


14.9 


603 


58.1 


16-30 


366 


35.4 


124 


12.0 


31 -45 


208 


20.1 


43 


4.1 


Over 45 hours per week 


63 


6.1 


55 


5.3 


Total 


1033 


100.0 


1037 


100.0 



So, one might ask if outside time demands compete with study time. To evaluate this question a series 
of correlations is reported. Each of the three correlations are statistically significant. The correlation 
between study time and employment is .269. The positive correlation indicates that as study time goes 
up, so does employment time. The size of the correlation is weak, but significant. The same applies to 
the other two correlations between employment and household work (r=.379) and between household 
work and study (r=.453). If employment detracted from study one would expect a negative correlation. 
So, the following correlations table would indicate that those students who report the greater study time, 
also report greater employment and greater household work commitments. This relationship is indicative 
of the non-traditional adult learners among students. Since the sum of these three time commitments 
does not equal the total available time, one may conclude that among students who report lower study 
times, they also report lower employment and household commitments. It appears possible to conclude 
that social commitments absorb the time not otherwise reported. (However, it should also be noted that 
in later examination, study time is minimally related to general education skills. See page 22) 



Spearman’s rank order correlations 


Average class 
study time 


Employment 

(volunteer) 


Household work 


Employment 

(volunteer) 


Correlation 

Coefficient 


.269 


1.000 






Sig. (2-tailed) 


.000 








N 


1028 


1033 




Household work 


Correlation 

Coefficient 


.453 


.379 


1.000 




Sig. (2-tailed) 


.000 


.000 






N 


1031 


1032 


1037 



Sixty-four percent of the respondents reported frequent or daily use of a computer to complete their 
course work with only 7.7% reporting limited or no use of a computer. 



Computer use 


Frequency 


Percent 


Daily 


290 


28.3 


Frequently 


368 


36.0 


Occasionally 


287 


28.1 


Limited 


64 


6.3 


No use 


14 


1.4 


Total 


1023 


100.0 



Among students participating in the assessment 85% reported English as their first language. 



English as first language 


Frequency 


Percent 


No 


168 


14.7 


Yes 


975 


85.3 


Total 


1143 


100.0 
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The characteristics of participants in this assessment indicate a broad variety of students. The inherent 
representativeness of these data then present both a rich potential for research and a challenge to 
simplify that complexity sufficiently to allow for focused assessment and evaluation. This examination 
will focus on comparing a freshmen and sophomores cross-section to national norms, then on change 
between freshmen and sophomores, and finally on what factors may impact the development of general 
education skills. 



College of DuPage students compared with national norms — Freshmen 

The challenge of comparing College of DuPage students with national norms lies in identification of 
appropriate cohorts and cross-sectional samples. The entry cohort — freshmen — can be selected from 
students entering College of DuPage with no previous college level credits. In this analysis 213 students 
were identified as entering freshmen. However, this is not a sufficient selection criterion since some 
students entering may not persist for two years. Persisting freshmen were identified as students in the 
fall cohort of test takers who identified themselves as beginning their college studies at College of 
DuPage, who had earned less than 15 quarter hour credits at the time of the test and who were enrolled 
in the Spring, 1999 quarter. These criteria eliminated “drop-outs" and school leavers. This group of non- 
persisters is labeled as “leavers” to avoid the connotation and assumptions about “dropping out.” In 
contrast to these negative stereotypes, several of the following analyses directly contradict such 
assumptions, with leavers sometimes scoring higher than persisters and graduates. 

A traditional assumption, perhaps incorrect, might be that leavers may be primarily lower achievers. If 
this (or the opposite) is the case, then a bias is introduced in comparing entering cross-sections with 
persisting cross-sections. Such a bias might be labeled a “freshmen survival effect." To avoid this bias, 
the analysis utilizes only entry level students who persist to the next round of testing. Of the 213 
students categorized as entering freshmen, 151 (71%) persisted to spring quarter enrollment and 62 
(29%) did not persist. In the academic year 2000 analysis, it may also be possible to examine the entry 
cohort persisting for two years. 

Throughout this and the following sections, comparisons between averages are reported in one or both of 
two ways. Each table simplifies and summarizes the comparison by listing the area-test and its relative 
position as statistically significantly higher, not statistically significantly different, or statistically 
significantly lower. These conclusions are based on calculated confidence intervals at the .95 level of 
significance. A second section of the table contains a listing of the (mean) average for each of the area- 
tests. Gray cells indicate no statistically significant difference between scores with a footnote reporting 
the probability of type I error based on a t-test for difference in means. T-tests were performed on 
comparisons between cohorts in the College of DuPage student research population, but not for 
comparisons of those cohorts and the national norms. 

Comparing freshmen-persisters with freshmen-leavers, there were no statistically significant differences 
on any of the area-test scores. In most of these comparisons, the means of leavers and persisters are 
very close (less than 1 standard error). In three comparisons, Writing Skills, Essay Writing, and 
Mathematics, the mean score differences were larger and might obtain significance with additional 
cases. In the comparison of Writing Skills, the mean of leavers was higher than that of persisters. In the 
comparison of Essay Writing and Mathematics, the scores of leavers were lower than that of persisters 
as might support a general assumption of least qualified students “dropping-out.” 
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Entering freshmen who persist compared with those who leave prior to next test round (Fall 
test-takers not enrolled in Spring with fewer than 15 hours at time of test who report beginning their 
studies at College of DuPage.) 


Persisters higher 














No significant 
difference 


Writing 

Skills 


Essay 

Writing 


Mathematics 


College 

Reading 


Critical 
Thinking. y? 


Science 

Reasoning 


Leavers higher than 
persisters 














Freshmen leavers 


:-6i:o 


2.93 


55.30 : 


59.25 


59.9:? 


57.25 ??|| 


Freshmen persisters 


59.97 


3.27 1 


57:57 


60.29 


60. It??? 


Ip7: 68 



The scores from the 151 persisting-freshmen were compared with the national norms published by ACT. 
In this comparison, the College of DuPage persisting freshmen cohort scored significantly higher on two 
area-tests — Essay Writing and Mathematics. On all of the other area-tests average scores were not 
statistically different from the 2-year public community college norms. 



Persisting freshmen compared with 2-year public community colleges 


C.O.D. freshmen 
higher 




Essay 

Writing 


Mathematics 








No significant 
difference 


Writing : & 
Skills ?| 






College 

Reading 


Critical. 

Thinking;?' 


^Science ?;x|f 
Reasoning 


C.O.D. freshmen 
lower 














C.O.D. freshmen 


59.97 - 


3.27 


57.57 


60,29 


60.11 :.?1| 


57.68 'i.?? 


2-year public 
community college 


61.0 


3.0 


55.8 


59.1 


58.7 


57.3 ' ?-? 



’In a t-test the native fresh men persister average is higher than the native freshmen leaver 
average with a probability of Type I error of .064. 
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53 - 


Writing Skills 


Mathematics 


College Reading 


Critical Thinking 


Science 

Reasoning 


■ COD Freshmen: significant 




57.57 








gCOD Freshmen: no difference 


59.97 




60.29 


60.11 


57.68 


■ 1998 Fresh, national norm 


61 


55.8 


59.1 


58.7 


57.3 



After publication of the Fall Quarter - 1998 freshmen scores, some faculty responses posited that our 
standard should be freshmen entering 4-year public colleges and universities. As an examination of the 
following chart supports, comparison to 4-year norms shifts most of the conclusions about significant 
differences down by one step. While the College of DuPage Essay Writing scores remain significantly 
higher than the 4-year public college norms, persisting- freshmen demonstrated no statistical difference 
from 4-year public college students in Mathematics. On the other four area-tests, their skills are 
statistically significantly below 4-year public college students. It is important to note, however, that the 
College of DuPage sample, like the public community college norm, contains both students intending to 
transfer and those pursuing terminal degrees, and thus the comparison to 4-year norms is less 
appropriate than the 2-year norms. 



Persisting freshmen compared with 4-year public colleges and universities 


C.O.D. freshmen 
higher 




Essay 

Writing 










No significant 
difference 






Mathematics i 








C.O.D. freshmen 
lower 


Writing 

Skills 






College 

Reading 


Critical 

Thinking 


Science 

Reasoning 


C.O.D. freshmen 


59.97 


3.27 


:;57:5T =V ' ; . 


60.29 


60.11 


57.68 


4-year college & 
university freshmen 


65. 


3.0 


58.9 


63.1 


63.3 


60.7 



Sophomores: 

As previously mentioned, the challenge of identifying which students among the general sample should 
be classified as sophomores is greater than that of identifying freshmen. The selection of completing 
sophomores utilized in this analysis includes cases identifying themselves as beginning their studies at 
College of DuPage and having earned between 80 and 1 10 credits by the time they took an area-test. 
These criteria resulted in 92 cases being selected. Of these cases, 25 were tested in the fall cycle and 
67 completed their assessment in the spring. None of these 92 cases was a duplicate or retest. 
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The College of DuPage completing sophomores cohort displayed average area-test scores higher than 
national norms in the areas of Mathematics and Science Reasoning. The completing-sophomores' 
average college Reading score is notably lower than the comparable 2-year communjty college 
sophomores scores. On the remaining three area-tests the completing sophomores’ averages are not 
statistically significantly different from the national norm. 



Completing sophomores compared with 2-year public community college sophomores 


C.O.D. sophomores 
higher 






Mathematics 






Science 

Reasoning 


No significant 
difference 


Writing 

Skills 


Essay 

Writing 




College 

Reading 2 


Critical 

Triinking 




C.O.D. sophomores 
lower 














C.O.D. sophomores 


62.81 


3.23 


62.67 


59.06 3 


62.24 


61.39 


2-year public 
community college 
sophomores 


62.7 


3.1 


56.1 


61.1 


81.2 


59.0 




I CoD Soph: significant 



62.67 



59.06 



61.39 



g CoD Soph: no difference 



62.81 



62.24 



11998 Soph. National Norm 



62.7 



56.1 



61.1 



61.2 



59 



As with the freshmen, some people may be interested in how the completing sophomores cohort 
compares with 4-year college sophomores. The Mathematics performance remains statistically 
significantly higher than the 4-year sophomores norms, although the Science Reasoning is not 
statistically significantly different. The college Reading performance remains statistically significantly 
lower while the other three area-tests are not statistically significantly different. 



2 Statistical significance for this section of a table was set at the 95% confidence level. The 
difference fails to meet this standard. However, the difference does satisfy other acceptable standards 
fora statistically significant difference. 

3 t-test = -1 .876: p <= .0789 
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Completing sophomores compared with 4-year public college and university sophomores 


C.O.D. sophomores 
higher 






Mathematics 








No significant 
difference 


Writing 

Skills 


Essay 

Writing 






Critical 

Thinking 


Science 

Reasoning 


C.O.D. sophomores 
lower 








College 

Reading 






C.O.D. sophomores 


62.81 


3.23 


62.67 


59.06 


62.24 


61.39 . 


4-year public college 
& university 
sophomores 


64.5 


3.2 


57.9 


. 63.0 


62.8 


60.7 . 



Students’ progress over the course of studies 

If the general conclusion from the first set of considerations is that College of DuPage students are not 
generally different from other 2-year community college students either as they begin or as they end their 
studies, then the next issue is to consider if they change over the duration of their studies. To examine 
this issue, a comparison is made between the cohort of entering-persisting freshmen, completing- 
sophomores earning 80 to 1 15 quarter hour credits, and graduating sophomores. 

The following table compares native-persisting-freshmen with sophomores completing 80-115 credits. 

In results from two area-tests, Mathematics and Science Reasoning, sophomores scored statistically 
significantly higher than freshmen (based on a statistically significant differences in both t-test and 
confidence intervals). In the Writing Skills comparison, the difference in averages may be accepted as 
showing an improved performance (despite not attaining statistically significance in confidence intervals 
but significant differences based on a t-test). 

In the remaining three area-tests, the scores are not statistically significantly different. In Critical 
Thinking, the cohort averages differ in an appropriate direction. On the remaining two area-tests, Essay 
Writing and college Reading, the averages for sophomores are lower than those for freshmen. Some 
part of this lack of statistically significant differences lies in the small number of cases in each area-test 
cohort. The freshmen cohort counts range from 22 (in Science Reasoning) to 31 (in Writing Skills). The 
sophomores cohort counts range from 10 (in Essay Writing) to 23 (in Science Reasoning). 
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